Skip to content

feat(cli): modularize CLI with noun-verb commands, --json output, and auto-discovery#445

Open
yeldarby wants to merge 44 commits intomainfrom
apr-2026/cli-modernization
Open

feat(cli): modularize CLI with noun-verb commands, --json output, and auto-discovery#445
yeldarby wants to merge 44 commits intomainfrom
apr-2026/cli-modernization

Conversation

@yeldarby
Copy link
Copy Markdown
Contributor

@yeldarby yeldarby commented Apr 2, 2026

Summary

Decomposes the monolithic 656-line roboflowpy.py into a modular roboflow/cli/ package with auto-discovered handler modules, global --json output for AI agents, and 17 command groups following a consistent noun verb pattern.

This is the foundational prep work for making Roboflow accessible to coding agents (CLI-first, MCP as thin adapter on top).

What changed

  • New roboflow/cli/ package with auto-discovery: drop a handler file in handlers/, it registers automatically
  • 17 command groups: auth, workspace, project, version, image, annotation, model, train, infer, workflow, deployment, batch, search, universe, folder, video, completion
  • Global --json flag works in any position (roboflow project list --json or roboflow --json project list) — outputs stable JSON schemas for agent consumption
  • Global --workspace and --api-key flags inherited by all subcommands
  • Universal resource shorthand: project, workspace/project, project/3, workspace/project/3 — version numbers always numeric for disambiguation
  • Backwards-compat shim: roboflowpy.py re-exports main and _argparser; all legacy command names (login, upload, download, search-export, upload_model, etc.) work as hidden aliases with exact original flag signatures
  • Custom _CleanHelpFormatter hides legacy aliases from --help while keeping them functional
  • suppress_sdk_output() prevents SDK "loading..." noise from corrupting --json output
  • Structured errors: output_error() with consistent {"error": {"message": "...", "hint": "..."}} schema, centralized JSON parsing to prevent double-encoding
  • Security: Config file written with 0600 permissions; API keys use params= not URL strings in new code
  • Deployment handler rewrite: Clean kebab-case names (create, machine-type, usage), legacy snake_case names as hidden aliases

What's implemented vs stubbed

Implemented Stubbed (help text registered, handler says "not yet implemented")
auth, workspace, project, version, image, model, train, infer, deployment, search workflow (build/run/deploy), batch processing, universe search, folder CRUD, completion, video status

Testing

  • 283 tests (185 new + 98 original), all passing
  • 3 rounds of QA with fresh-user perspective, 5/5 ratings across discoverability, consistency, agent-friendliness, error quality, and overall polish
  • PR review with reviewer + EM sign-off
  • Security review — no critical/high findings, 2 medium items fixed
  • make check_code_quality clean (ruff format, ruff check, mypy)

How to test

pip install -e ".[dev]"
roboflow --help
roboflow --json project list
roboflow --version
python -m unittest
make check_code_quality

Backwards compatibility

  • setup.py entry point unchanged: roboflow=roboflow.roboflowpy:main
  • from roboflow.roboflowpy import main and _argparser still work
  • All old command names (login, whoami, upload, import, download, train, search-export, upload_model, get_workspace_info, run_video_inference_api) work identically
  • All old deployment subcommands (add, machine_type, usage_workspace, usage_deployment) work with original flags

yeldarby and others added 30 commits April 1, 2026 14:37
Create the roboflow.cli package with auto-discovery of handler modules,
global --json/--workspace/--api-key/--quiet flags, output helpers for
structured JSON and human-readable formatting, a resource shorthand
resolver for workspace/project/version addressing, and test scaffolding.

Replace the monolithic roboflowpy.py with a backwards-compat shim that
delegates to the new roboflow.cli.main(). The setup.py entry point
remains unchanged.

This is the foundation for the CLI modularization effort. Handler modules
will be added in parallel by separate engineers in Wave 1.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements auth (login, status, set-workspace, logout) and workspace
(list, get) commands following the modular handler pattern. Includes
unit tests for registration and argument parsing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
QA found that `roboflow --json --version` output plain text instead of
JSON. Now outputs `{"version": "1.2.16"}` when --json is active.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement project (list, get, create) and version (list, get, download,
export, create stub) subcommands following the handler pattern with
lazy imports, output() for JSON support, and resolve_resource() for
shorthand parsing. Includes unit tests for arg parsing and registration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A broken handler module must not take down the entire CLI. Wrap the
auto-discovery import+register in try/except and log failures at debug
level. This unblocks testing of working handlers while others are still
being developed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement image handler with upload (single + directory), get, search,
tag, delete, and annotate commands. Add annotation handler with stub
commands for batch and job operations (3-level nesting).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…erse, folder, batch, completion

Migrates deployment (thin wrapper around existing roboflow.deployment with create/machine-type aliases),
search (workspace search + export), and video infer from old CLI. Adds stub handlers for workflow,
universe, folder, batch, and completion commands. Includes unit tests for all 8 handlers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement three new CLI handler modules for the modernized CLI:
- model.py: list/get/upload commands for trained model management
- train.py: start training with backwards-compat (train == train start)
- infer.py: top-level inference command with auto type detection

All model class imports are lazy. All commands support --json output.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace _subparsers._group_actions traversal with _actions iteration
using isinstance check, which is more robust across Python versions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace Roboflow()/workspace() SDK calls in _list_projects with direct
rfapi.get_workspace() to avoid "loading Roboflow workspace..." messages
on stdout. Also suppress stdout in _create_project when --json or
--quiet is active.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bug 1: Non-interactive login (--api-key) crashed because the API
validation endpoint returns a welcome message, not workspace data.
Now stores API key directly with workspace URL from the response.

Bug 2: auth status failed after set-workspace because it read from
get_conditional_configuration_variable which didn't see the config
file updates. Now reads directly from the config file via _load_config.

Bug 3: workspace get with invalid ID showed raw Python traceback.
Now catches RoboflowError and shows a clean error message.

Rough-edge: workspace get now shows human-readable text in non-JSON
mode instead of dumping raw JSON.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire up _aliases.py with all top-level convenience commands:
- roboflow login → auth login
- roboflow whoami → auth status
- roboflow upload → image upload
- roboflow import → image upload (directory)
- roboflow download → version download
- roboflow search-export → search --export
- roboflow upload_model → model upload
- roboflow get_workspace_info → preserved compat handler
- roboflow run_video_inference_api → video infer

Also includes compatibility fix in image.py to handle attribute name
differences between canonical handlers and alias parsers.

246 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Validate --add/--remove required before executing tag command
- Annotation stubs output JSON error when --json is active
- Improve upload path help text to mention auto-detection
- Handle alias arg name differences (tag_names/tag, num_retries/retries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. auth status now falls back to --api-key flag and ROBOFLOW_API_KEY
   env var when no config file exists, fetching workspace info from
   the API instead of reporting "Not logged in."

2. Non-interactive auth login now fetches the real workspace name via
   rfapi.get_workspace and shows a note that API key login only stores
   the key's workspace (vs interactive login which gets all).

3. workspace get text output handles members as int (API returns count)
   instead of assuming it's a dict/list.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix search handler stdout corruption in --json mode by redirecting
  SDK "loading..." messages when --json or --quiet is active
- Hide legacy alias commands (upload_model, get_workspace_info,
  run_video_inference_api, search-export) from help output
- Add missing --no-extract help text in search-export alias
- 248 tests pass, all linting clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- deployment: set default func so `roboflow deployment` shows its own
  subcommand help instead of top-level CLI help
- deployment: wrap all legacy handlers with _wrap_deployment_func to
  intercept bare print()+exit() and produce structured JSON errors
  with normalised exit codes (<=3)
- search: suppress "loading Roboflow workspace..." in --quiet mode too
  (was only suppressed in --json mode)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- model list: wrap SDK calls in try/except for clean error output
- model/train: extract message from JSON-encoded API errors to prevent
  double-encoding in --json mode
- Add tests for error message extraction and model list 404 case

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a context manager to _output.py that redirects stdout when --json or
--quiet is active, preventing SDK "loading..." messages from corrupting
structured output. Used by model and search handlers.

256 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Records 5 deviations from the original CLI modernization plan,
including the rationale and assessment for each change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…tubs

1. Add _CleanHelpFormatter that filters SUPPRESS-ed subparser choices
   from both the {choices} usage line and the command list. Hidden legacy
   aliases (upload_model, get_workspace_info, run_video_inference_api,
   search-export) are now truly invisible in --help output while still
   being functional.

2. Fix annotation stubs to use output_error() instead of bare print(),
   making them exit with code 1 (consistent with all other stubs) and
   produce proper JSON in --json mode.

256 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Default --annotation to the project name when not provided, so
  `project create` works without the flag (the API requires it).
- Parse HTTP 422 response bodies to surface actionable error hints.
- Add human-readable key-value output for `project get` instead of
  dumping raw JSON in non-JSON mode.
- Use suppress_sdk_output context manager in create_project.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Created/Updated fields now display as human-readable dates
(YYYY-MM-DD HH:MM:SS) instead of raw epoch floats.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… mode

When an API error message is itself a JSON string, output_error now
parses it so the error field contains a proper object instead of a
stringified JSON string. This lets agents parse errors with a single
json.loads() call.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move _extract_error_message logic from model.py and train.py into
_parse_error_message in _output.py. Now output_error automatically
handles JSON-encoded API errors for all handlers: parsed objects in
--json mode, human-readable messages in text mode. Removes duplicate
helpers and updates tests to use the centralized function.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… handlers

- image upload (single and directory): wrap SDK init in
  suppress_sdk_output, add try/except with output_error for both
  initialization and upload operations
- version download: wrap Roboflow() and workspace/project calls in
  suppress_sdk_output to prevent "loading..." noise in --json mode

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on download

Wrap all SDK calls (including single_upload, upload_dataset, versions,
download) inside suppress_sdk_output context to prevent "loading..."
noise from corrupting --json output. Also consolidate try/except to
catch errors from SDK operations, not just initialization.

256 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…download

Replace conditional suppress_sdk_output (only active in --json/--quiet)
with unconditional redirect_stdout during workspace/project initialization.
SDK "loading Roboflow workspace/project..." messages are implementation
details that should never appear in CLI output regardless of mode.

Also separate init try/except from operation try/except so errors during
workspace/project loading get exit_code=3 (not found) while upload errors
get exit_code=1 (general).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. suppress_sdk_output now always suppresses SDK "loading..." messages
   in all modes (not just --json/--quiet). These messages are SDK noise,
   not CLI output. The CLI controls its own output via output()/output_error().

2. Deployment handler improvements:
   - "create" alias uses hyphenated flags (--machine-type, --no-delete-on-expiration)
   - "create" alias no longer has its own -a flag; uses global --api-key
   - "machine-type" alias has clean help text
   - Wrapper now properly emits structured JSON for success responses in --json mode

3. All 256 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…le-nested errors

- Reorder global flags (--json, --workspace, etc.) before argparse parsing
  so they work in any position (e.g. `roboflow project list --json`)
- Replace `classification` with `single-label-classification` and
  `multi-label-classification` in project create --type choices to match API
- Unwrap double-nested error JSON: `{"error":{"error":{...}}}` is now
  `{"error":{...}}`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
batch create now shows --workflow, --input, --model, --output flags.
batch list accepts --status filter. batch results accepts --format.
Helps users understand the planned interface even before implementation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
yeldarby and others added 10 commits April 1, 2026 21:04
- whoami/auth status now validates against the API when --api-key is
  explicitly provided, instead of silently showing saved config
- Error JSON output is now always {"error": {"message": "...", ...}}
  instead of sometimes a string, sometimes an object — consistent
  schema for AI agents and programmatic consumers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…idden

Replace the delegation to add_deployment_parser() with a fresh set of
subcommands using clean kebab-case names:
- machine-type (was machine_type)
- create (was add)
- usage (merges usage_workspace + usage_deployment)
- get, list, pause, resume, delete, log (unchanged)

New commands use:
- --machine-type not -m/--machine_type
- --email not -e/--creator_email
- --no-delete-on-expiration not -nodel
- --inference-version not --inference_version
- --wait not -w/--wait_on_pending
- No -a flag (uses global --api-key)
- No -t flag reuse (--duration and --to use long form only)

Legacy snake_case names (add, machine_type, usage_workspace,
usage_deployment) are registered as hidden aliases with SUPPRESS help
and their exact original flag signatures, so existing scripts keep
working unchanged.

Uses _CleanHelpFormatter on the deployment subparser to hide legacy
aliases from the {choices} line.

259 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When using ROBOFLOW_API_KEY or --api-key without having run `roboflow auth login`,
`workspace list` now queries the API to resolve the workspace instead of
showing empty results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added resolve_default_workspace() helper to _resolver.py that queries
the API validation endpoint when RF_WORKSPACE is not in config. Used by
project list, project get (short slug), and workspace list so all commands
work consistently with just ROBOFLOW_API_KEY set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When PIL cannot identify an uploaded file, the error now includes a hint
listing supported image formats instead of just the raw exception message.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ency

- Fix download alias crash: use url_or_id as dest with datasetUrl metavar
- Add return after output_error in image.py for static analysis safety
- Replace bare print in version create stub with output_error
- Standardize SDK suppression to suppress_sdk_output() everywhere
- Extract 7 identical _stub functions to shared stub() in _output.py
- De-duplicate redundant os.getenv("ROBOFLOW_API_KEY") in workspace.py

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ests

- Use requests params dict for api_key instead of embedding in URLs (image.py)
- Replace min(code, 3) with explicit exit code mapping in deployment.py
- Add unit tests for _reorder_argv edge cases (12 tests)
- Add backward-compat alias tests including download regression (11 tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…iscovery.py

Move _reorder_argv and backward-compat alias tests into test_discovery.py
per EM direction. Remove separate test_reorder_argv.py and test_aliases.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Config file now written with 0600 permissions (owner read/write only)
   instead of default 0644. Prevents other users on shared systems from
   reading stored API keys from ~/.config/roboflow/config.json.

2. Login alias --api-key flag now uses dest="login_api_key" to match
   what _login() handler reads, fixing a dead code path where the
   alias's --api-key value was silently ignored.

278 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The old roboflowpy.py exported _argparser and other functions that
external scripts (including tests/manual/debugme.py) may import.
Re-export _argparser as an alias for build_parser so existing code
like `from roboflow.roboflowpy import _argparser` continues to work.

Add dedicated backwards-compatibility test suite verifying the shim
exports, parser construction, and legacy command name parsing.

283 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@yeldarby yeldarby requested a review from tonylampada April 2, 2026 03:13
@yeldarby
Copy link
Copy Markdown
Contributor Author

yeldarby commented Apr 2, 2026

Re: CodeQL security scan findings on _output.py

All 5 findings from github-advanced-security[bot] are false positives — CodeQL traces data flow from API responses through to print() and flags them as "clear-text logging of sensitive information."

These print() calls are the CLI's intentional output mechanismoutput() prints to stdout (the whole point of a CLI), and output_error() prints to stderr. This is not logging; it's the user-facing interface.

Mitigation already in place:

  • API keys are masked via _mask_key() in auth.py before reaching output() (e.g., auth status shows tV****************M7)
  • --json error output goes to stderr (not stdout) to prevent sensitive data leaking into piped workflows
  • Config file is written with 0600 permissions
  • New code uses params={"api_key": ...} instead of URL string interpolation

No action needed on these findings.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: adfa65645c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

yeldarby and others added 2 commits April 1, 2026 22:27
1. Remove -w from _reorder_argv global flags — it collides with
   deployment's -w/--wait_on_pending (boolean). --workspace long form
   is still reordered safely. Prevents deployment add -w from being
   misinterpreted as workspace flag.

2. Forward annotation_group to workspace.search_export() in _do_export.
   Also add -g/--annotation-group flag to the canonical search command
   (was only on the hidden search-export alias).

3. Restore -M shorthand on upload alias for backwards compat with
   scripts using `roboflow upload ... -M '{"key":"val"}'`.

283 tests pass, all linting clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- CLAUDE.md: Replace old CLI section with full modular architecture docs
  (package structure, handler pattern, agent experience requirements,
  documentation policy)
- CLI-COMMANDS.md: Rewrite as concise quickstart linking to docs.roboflow.com
  for full reference. Covers install, global flags, common examples, --json
  for agents, resource shorthand, backwards compat table.
- CONTRIBUTING.md: Add CLI Development section with handler template,
  agent experience checklist, and documentation policy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@tonylampada tonylampada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!
maybe increase a minor (1.3.0)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants